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Abstract 

In this work, a new upper bound for average error probability of a two- 
user discrete memoryless (DM) multiple-access channel (MAC) is derived. 
This bound can be universally obtained for all discrete memoryless MACs 
with given input and output alphabets. This is the first bound of this 
type that explicitly uses the method of expurgation. It is shown that the 
exponent of this bound is greater than or equal to those of previously 
known bounds. 
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1 Introduction 



A crucial problem in network information theory is determining the average 
probability of error that can be achieved on a discrete memoryless multiple- 
access channel. More specifically, a two-user DM-MAC is defined by a stochastic 
matri50 W : X x y ^ Z, where the input alphabets, X, y, and the output 
alphabet, Z, are finite sets. The channel transition probability for sequences of 
length n is given by 

n 

W"{z\^,y)^Y[Wiz,\x„y,) (1) 

i=l 

where 

x^(xi,...,x„)GA'",y^(2/i,...,2;„)e3^" 

and 

z^(zi,...,2„)ez". 

It has been proven, by Ahlswede [1] and Liao's [11] coding theorem, that for 
any {Rx,Ry) in the interior of a certain set C, and for all sufficiently large 
n, there exists a multiuser code with an arbitrary small average probability of 
error. Conversely, for any {Rx,Ry) outside of C, the average probability of 
error is bounded away from 0. The set C, called capacity region for W , is the 
closure of the set of all rate pairs {Rx, Ry) satisfying [15] 

Q <Rx <I{X hZ\Y,U) (2a) 
Q <Ry <I{Y ^Z\X,U) (2b) 
Q<Rx+Ry <I{XY hZ\U), (2c) 

for all choices of joint distributions over the random variables ?7, X, Y, Z of the 
form p{u)p{x\u)p{y\u)W{z\x,y) with U E U and \U\ < 4. As we can see, this 
theorem was presented in an asymptotic nature, i.e., it was proven that the error 
probability of the channel code can go to zero as the block length goes to infinity. 
Yet, it does not tell us how large the block length must be in order to achieve a 
specific error probability. On the other hand, in practical situations, there are 
limitations on the delay of the communication. Additionally, the block length 
of the code cannot go to infinity. Therefore, it is important to study how the 
probability of error drops as the block length goes to infinity. A partial answer 
to this question is provided by examining the error exponent of the channel. 

Error exponents have been meticulously studied for discrete memoryless 
channels in point to point data communications. Lower and upper bounds 
are known on the error exponent of these channels. A lower bound, known as 

^We use the following notation throughout this work. Script capitals U, X, y, Z,. . . denote 
finite, nonempty sets. To show the cardinality of a set X, we use \X\. We also use the letters 
P, Q,. . . for probability distributions on finite sets, and U, X, Y,. . . for random variables. 
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the random coding exponent, was developed by Fano [8]. The random coding 
bound in information theory provides a well-known upper bound for the proba- 
bility of decoding error of the best code, of a given rate and block length. This 
bound is constructed by upper-bounding the average error probability over an 
ensemble of codes. Gallager [6] demonstrated that the random coding bound 
is the true error exponent for the random code ensemble. This result illus- 
trates that the weakness of the random coding boimd, at low rates, is not due 
to upper-bounding the ensemble average. Rather, this weakness is due to the 
fact that the best codes perform much better than the average, especially at 
low rates. Barg and Forney [2] investigated two different upper bounds on the 
average probability of error, called the typical random coding bound and the 
expurgated bound. The typical bound is basically the typical performance of 
the ensemble. By this, we mean that almost all random codes exhibit this per- 
formance. In addition, they have shown that the typical random code performs 
much better than the average performance over the random coding ensemble, 
at least, at low rates. The random coding exponent may be improved at low 
rates by a process called "expurgation" which yields a new bound that exceeds 
the random coding bound at low rates. It has been shown that the expurgated 
bound is strictly larger than both the random coding and the typical random 
coding bounds at low rates. It has also been demonstrated that both the ex- 
purgated and the typical random coding bounds are equal at i? = 0. At this 
specific rate, the upper bound on the reliability function is also equal to these 
bounds [4, pg. 189]. 

In regard to the Multiple-Access Channels, stronger versions of Ahlswede 
and Liao's coding theorem, giving exponential upper and lower bounds for 
the error probability, have been derived by numerous other authors. Slepian 
and Wolf [15], Dyachkov [5], Gallager [7], Pokorny and Wallmeier [14], and 
Liu and Hughes [12] have all studied upper bounds on the error probability. 
Haroutunian [10] and Nazari [13] studied lower bounds on the error probabil- 
ity. The random coding bound for MAC was studied by Gallger [9], Pokorny 
and Wallmeier [14], and Liu and Hughes [12]. In this paper, we mostly con- 
centrate on the result of [14] and [12]. Both of these random coding theorems 
are universal, i.e., a fixed choice of codewords and decoding sets achieve their 
upper bounds for all MACs with given input and output alphabets. In deriving 
both bounds, three crucial steps are observed. The first step is the choice of 
the ensemble. In [14], each codeword of each code in the ensemble is chosen 
from Tp^ and Tp^, for some Px and Py- However, in [12] for a fixed distri- 
bution, PuPx\uPy\Uj the codewords of each code in the ensemble are chosen 
from Tp^i^^(u) and Tpy^^{u) for some sequence u G Tp^. The second step is 
the packing lemma, in which the existence of some particular code with cer- 
tain properties is proven. The way the existence of such a code is proved is 
through random coding argument over the ensemble. As a side result of this 
step, it can be shown that most codes in the ensemble of [14] [12] have these 
properties. In the third step, an appropriate decoding rule is first chosen, and 
the performance of the code, found in the packing step, is analyzed. It has 
been shown that the result of Liu and Hughes is tighter than Pokorny's since 
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they used a different ensemble and a differnet decoding rule. In this work, we 
follow a similar three-step approach. First, we start with an ensemble identical 
to [12]. Then, we provide a new packing lemma in which the resulting code has 
more constraints in comparison to the packing lemmas in [14] and [12]. This 
packing lemma is very similar to Pokorny's packing lemma, in the sense that 
only channel inputs appear in the packing inequalities. One of the advantages 
of this packing lemma, in comparison to [14], is that it enables us to partially 
expurgate some of the codewords and end up with a new code with stronger 
properties. In general, expurgation has not been studied in MAC, since by 
eliminating some of the codeword pairs, we may end up with correlated input 
sequences. In this work, we do not eliminate pairs of codewords. Rather, we 
expurgate codewords from only one of the codebooks. Finally, we analyze the 
performance of the expurgated code and end up with a new upper bound on 
the probability of error. 

This paper is organized as follows: section II introduces terminology, and 
section III summarizes our main results. The proofs of some of these results are 
given in the Appendix. 



2 Preliminaries 

For any alphabet X, V{X) denotes the set of all probability distributions on 
X. The type of a sequence x = {xi,...,Xn) € Af" is the distributions Px on X 
defined by 

P^{x) 4 -Nix\x), xeX, (3) 

n 

where A^(x|x) denotes the number of occurrences of x in x. Let Vn{X) denote 
the set of all types in X"' , and define the set of all sequences in X" of type P as 

Tp 4 {x e A"" : Px = P}. (4) 

The joint type of a pair (x, y) G A*" x 3^" is the probability distribution Px,y 
on X xy defined by 

Px,y(a;, y) = ^N{x, y\x, y), (x, y)eXxy, (5) 

where A''(x,y|x,y) is the number of occurrences of {x,y) in (x, y). The rela- 
tive entropy or Kullback-Leibler distance between two probability distribution 
P,Q e V{X) is defined as 

D{P\\Q)^Y.P{x)log^ (6) 
xex 

Let W(y|<V) denote the set of all stochastic matrices with input alphabet X 
and output alphabet y. Then, given stochastic matrices V, W € yV{y\X), the 
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conditional I-divergence is defined by 

DiV\\W\P) ^ J2 Pi^)DiV{-M\Wi-\x)). (7) 

Definition 1. An {n,M,N) multi-user code for a given MAC W, is a set 
{(xi, Yj, Aj) ■■l<i<M,l<j<N} with 

• n A'j' = /or ^ (z',/). 

Definition 2. When message {i,j) is transmitted, the conditional probability of 
error of the multiuser code C is given by 

e,,{C,W)^W^{D^,^\x,,y,). 

The average probability of error for multiuser code, C, is defined as 

M N 

3 main result 

In this section, we present a new, universally achievable upper bound on the 
average error probability of multiple-access channel. We observe that the mutual 
position of the codewords plays a crucial role in determining the decoding error. 
Intuitively, we expect that the codewords in a "good" code must be far from each 
other. In accordance with the ideas of Csiszar and Korner [4], we use conditional 
types to quantify this statement. Basically, we shall select a prescribed number 
of sequences in X" and 3^" so that the shells around each pair have small 
intersections with the shells around other other sequences. In general, we have 
two types of packing lemmas based on whether the output of the shell belongs 
to the channel input space or channel output space. The Packing lemma in [14] 
belongs to the first type, and the one in [12] belongs to the second type. All 
the inequalities in the first type depend only on the channel input sequences. 
However, in the second type, the lemma incorporates the channel output into 
the packing inequalities. In this work, we use the first type. In the following, 
we prove three packing lemmas. In lemma [TJ we show that there exists a good 
code with some certain properties. The nature of these properties is average, in 
the sense that they guarantee ,on the average, the codewords in the code are far 
from each other. One can easily show that by using this packing lemma and an 
appropriate decoder, all the results of [14] and [12] can be re-derived and unified. 
In lemma [21 we go one step further, by proving that the code found in lemma [1] 
has some additional properties that are now guaranteed for all individual pairs 
of sequences. If we use this packing lemma in bounding the average probability 
of error, we will get a tighter bound, especially at low rates. One can show that 



4 



most of the random codes from the ensemble have these properties. Hence, this 
kind of bound is called the typical random coding bound in accordance to [2] . 
Finally, In lemma [31 we use one of these typical codes and eliminate some of 
its codewords. The resulting code has all the previous properties mentioned in 
lemma [Hand lemma [5] In addition, this code satisfies some additional stronger 
constraints. In lemma[ll we show that only some of the joint type can be seen in 
the expurgated code. Finally, we calculate a new upper bound for the average 
probability of error, depending only on the properties of the set of codewords 
resulting from expurgation. 

Lemma 1. For every finite set U, Pxyu G 'P,i(Z^ x A:" x 3^) such that X — U — Y , 
Rx > 0, Ry > , (5 > 0, and u € Tp^, there exists sets of codewords Cx = 
{xi,X2, Xmx} and Cy = {Vi, 1/2, Vmy} ^» ^ ^Px\u^'^^' ^ '^Kiu^'^^ 
for all i and j, Mx > 2"^^ , and My > 2"^^ , such that for every joint type 
VjjxyxY e Pn(M X (A" X y)'^), whenever n > no{\U\, \X\, \y\,S), 

Mx My 



MxMy ^ ^ 

1=1 J=l 



^ Mx My 



MxMy^^^ -UXYY 
1=1 3 = 1 lyij 



< 2-n[FY{V)-3S] ^-^Q^ 

^ Mx My 

nrr- E E E i^v„,.^ K 2/„ 



MxMy^^^^^^ --uxYx 



^ 2-n[Fx{V)-3S] ^^-^^ 

^ Mx My 

Hiz: E E E E ^^^uxy^^ y^^^^^ vd 



MxMy ^-^ ^-^ ^-^ ^-^ 'UXYXY 
i=l j=l k^i l^j 



2-n[FxY{V)-'iS] ^-1^2) 



wher 



F{V) = IviX AY\U) (13) 

Fx{V) ^Iv{X A Y\U) + Iv{X A Y\U) 

+ Iv{X AX\UY)-Rx (14) 

Fy{V) ^ IviX a Y\U) + Iv{X A Y\U) 

+ IviY AY\UX)-Ry (15) 

Fxy{V) ^ IviX a Y\U) + Iv{X A Y\U) 

+ IviXY AXY\U) - Rx - Ry (16) 

Here U , X , Y , X , Y denote random variables with common distribution V^-^yxY ^ 
Vn{l^ X (A" X 3^)^), and VuxY, ^uxyX' and V^^yy are appropriate marginal 
distributions of V^-^yj^y . 
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Proof. In this proof, we use a similar random coding argument that J. Pokorny 
used in [14] . The main difference is that our lemma uses a different code ensem- 
ble which results in a tighter bound. Instead of choosing our sequences from 
Tp^ and Tp^ , we choose our random sequences uniformly from Tp^ ^ (u) , and 
Tp^^^{u) for a given u G Tp^. In [12], we see a similar random code ensemble, 
however, their packing lemma incorporates the channel output z into the pack- 
ing inequalities. One can easily show that, by using this packing lemma and 
considering the minimum equivocation decoding rule, we would end up with the 
random coding bound derived in [12]. □ 

Lemma 2. For every finite set U , PxYU G Vnipi x. X x. y) such that X — 
U — Y, Rx > 0, Ry > , S > 0, and u e Tp^, there exists sets of codewords 
Cx = {xi,X2,...,xmx} and Cy = {y^, 2/2, 2/4^^ } with x, e T^^^^{u), G 
T^^i^(ti) for all I and j, Mx > 2"-^^, and My > 2"^^, such that for every 
joint type Vjj-^yxy ^ ^n(^ x {X x y)'^), ([9])- (fT2|) are satisfied provided n > 
no{\V(\, \X\, \y\, S). Moreover, for any I < i < Mx, and any 1 < j < My 

Ir^^^^ (n, x„ y^) < 2--inv)-H.-R.-2S] (17) 

Y: 1t.^_, {u, ^, y,, X,) < 2-«[^x(v)-«.-«,.-3.] (18) 

It^ _ (n, X,, y^,y,) < -H.-n.-3S] (19) 

V'V'Itv .Au,x„y,Xk,yi) 

i—^i—^ UXYXY J 

^ 2-n[FxYiV)-Rx~'RY-4:5] (20) 

Proof. Let us use the result of lemma [TJ and multiply both sides of the inequal- 
ities dH)- (IE]) by MxMy. □ 

Lemma 3. For every finite set U, Pxvu G Vn{U x X x y) such that X — U — Y, 
Rx > 0, Ry > , d > 0, and u G Tp^ , there exists sets of codewords Cx = 
{a;i,a^j,...,a;M* } and Cy = {y^, y2, ■-, Vm^} ^ ^Px\u' ^ ^Py\u ^""^ 

all I and], M^ > 2"(«^-'5), and M^ > 2<^^-^\ such that for every joint type 
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VuXYXY e T'niU X (A- X yf), 

M\ My 

y y Itv {u, X,, yA < 2-"[^(^)-3*l (21) 

Mx My 



y i=i j=i k^i 

^ 2-n[FxiV)-4S] ^22) 

Mi M* 

E E E 1^-,,,^ ' y^) 

^ ^ j=i j=i 

< 2-"[-Pi'(^)-4<5] ^23) 

jiUE E E E E i^v^..^^- 2'.' 

A y i=i j=i fe^i 

2""[-PA'y(V)-5i5] ^24) 

and for any I < i < M'^, and any \ < j < My 

It.^^^ (n, x„ y^) < 2-"[^'(^)-«""{«-.«-}-35] (25) 
V It, _ (tt, X,, y^,Xk) < 2-"[^x(^)— (26) 



Y: It.^^y, [u, a,, y^, y,) < 2-"[^WV)-min(;..,;..}-4.] (27) 
EE^^v - ^{u,Xi,y,,Xk,yi) 



(7XyX^ 



whenever 

n>n^{\U\,\X\,\yl5) 
where F{V), Fx{V), Fy{V), Fxy{V) are defined m ^-1^. 

Proof. Let Cx = {xi, X2, xa^^ } and Cy = {Yi, y2> yA/i^} be the collec- 
tions of codewords whose existence is asserted in lemma [TJ From lemma [1] the 
codewords satisfy 

My , Mx 



_)_yj_y ^ 2-"[^(^)-2^1 . (29) 

My ^ Mx ^ -'^'c/xyV , ijjjj - y J 

j=i i=i 

Therefore, there exist My > codewords in Cy that satisfy 

T^El-v_("'-''y.)^2-"'"^''^-^'><2- (30) 



1=1 
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Let us call this set of codewords Cy. By multiplying both sides of ([50)1 with 
Mx, and considering the fact that all terms in the summation are nonnegative, 
it can be concluded that for every e Cx, Yj G Cy, 

1t.,,^ (u,x„y^.) < 2-"[^(^)-2^--«-l X 2. (31) 

We can make a similar argument and conclude that there exists a subset of Cx , 
called Cx, with Mjr > codewords such that for any x^ g Cj^, y^ g Cy 

lT,„^,(u,x„y,.) < 2-"[^(^)-2^-^-l X 2. (32) 

Without loss of generality, let us assume Rx < Ry- In this case, (|3ip will end 
up with a tighter result. Using (fTO|) . we conclude that 

Since M^>^, 



^ y y It, (u,x„y,,y,) < 2-"[^-(^)-3^1 x 2, 



again, by a similar argument, there exists My > codewords, y,-, in Cy such 



2 

that 



^ Mx 

< 2-"[^-(^)-3^] X 4. (33) 
Let us call this subset of Cy as Cy. Therefore, for any x^ G Cx, Yj G Cy, 

5:iT,,^^^^(u,x,:,y,,y,) < 2-«[^>'(^)-3^-«-l x 4. (34) 

By using (jlip. we can conclude that 



-n[_Fx(V)-35] 

MxMy 



Considering the fact that My > > we can conclude that 

I V V It fu X- V- Xt) < 2-"[-P^^(^)-3'51 X 4 

^ Y i&Cx k^i 
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Hence, there exists Cy C Cy, with My > codewords such that 



i£Cx k^i 

< 2-«[i=^x(y)-35] ^ 8 (35) 
Therefore, for all £ Cx, Yj £ Cy, 

V It, ^ (u, X,;, y^., Xfe) < 2-"[^-(^)-«--3*-l X 8. (36) 

Similarly, by using we can conclude that 

7m;: E EEi^w.t^^"''^-^^'^'^'^') 



MxMy -t^ f-' tf. --"^^^^ 



By a similar argument and using the fact that My > > > we 
conclude that 

E EEl^v„,.^^(u,x„y^,x,,y,) 

< 2-"[^^^(^)-3«l X 8. (38) 
Therefore, there exist Cy C Cy, with A/y > codewords, such that 

^ E EEKx.**-^"'^-^^'^^'^') 

< 2-"[^^^(^)-3'51 X 16. (39) 

Similarly, since My > > > > ^j^, we conclude that for all 
e Cx, y, G C|-, 



E E i^wx? y.' y^) 



UXYXY 

k^i l^J 



Since Cy C Cy C Cy C Cy^ any codeword belonging to Cy has all the properties 
we derived in ([3T|l . ([34]) . ([36|l . (|40| . Therefore, we have proven that there exists 
a codebook Cy C Cy with My > 4^ codewords such that for any x^ £ Cx, 
Yj S Cy, we have the properties ([^ . p4|) . ([36)) . l|40)) . As shown, we have 
eliminated some of the codewords from Cy- Similarly, we can do the expurgation 
on Cx- If Rx < Ry, the expurgation on Cy results in a tighter result. However, 
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if Rx > Ry , the expurgation on Cx would end up with a tighter bound. Thus, 
in general, there exists a pair of codebooks (C^, ), with |C^||Cf | > 
such that for any G C^, S C^, 

lT.„^^(u,x„y,) < 2-«[^^(^)— (41) 
^ 1t,^^^^ (u,x„y^-,xfc) < 2-«[^-(^)— (42) 

^1t,^^^_ (u,X,;,y^.,y;) < 2-"[^-(V)-min{/?.,fl.}-4^] (43) 



UXYXY 



The only difference between the exponents in ((25 |) - ([28|) and the ones in ([9])- 
is min{i?x,^y}- Despite of the ©-((T^) which are upper bounds for some 
quantities averaged over all pairs of sequences belonging to [Cx , ) , the results 
in (I25l)-(l28l) are valid for all pairs of codewords in (C|^,Cf ). Let us define = 
\C*x\, = \C^\. In the following, we will show that the new codebook pair, 
{Cxfiy), still satisfies the same average performance bound we obtained for 
the original codebook pair, (Cx,Cy). The functions in (f9|)- (fT2|) for the new 
codebook pair can be upperbounded as follows. 



M* Mt 



M*M* 

X y i=i j=i 

Mx My 



^ y i=i j=i 

Mx My 

^ ]^^^EEi^-.x.("'^-y.) 

4 = 1 j = l 

< i6*2-"[^(^)-35] < 2-"[^(^)-25]. (45) 
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We can use a similar argument and show that 

M' M' 



ITT E E E E ^^^u..^9 ' 



A Y i^i j^i fc-^i 



Mt Mv 



M* M* 5Z E E l^v.„,,^ (u, x„ y^, y,) 



i=l j=l 1^3 



I^P^T E E E l^v.„,,^ (u, x„ y^, X,) 

J i=l j=l k^i 

Here, by method of expurgation, we end up with a code with a similar average 
bound as we had for the original code. However, all pairs of codewords in the 
new code also satisfy (P5|) - (l^ . Therefore, we did not lose anything in terms of 
average performance, however, as we see in theorem [1], we would end up with a 
tighter random coding bound since we have more constraints on any particular 
pair of codewords in our codebook pair. □ 

Lemma 4. For any type Vjj-^yxY ^ x [X x yY) such that for some 

Xi, Xk G Cx, and y^.yi e Cy, 

(tt, a;,, yj,Xk, Vi) G Ty^^^^^ (49) 
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the following inequalities must be satisfied 

Vxu = = Pxu, Vyu = = P^ 



Iv{X AY 
IviX AY 
Iv{X AY 

IviX AY 

IviX AY 

IviX AY 

IviX AY 

IviX AY 



YU 

U)JviX A Y\U) < iam{Rx,RY} + 3(5 
U)JviX A Y\U) < mm{Rx,RY} + 3(5 
U) + IviX A Y\U) + IviX A X\UY) 

< i?x +niin{i?x,i?y} + 4(5 
U) + IviX A Y\U) + IviX A X\UY) 

< i?x +niin{i?x,i?y} + 4(5 
U) + IviX A Y\U) + IviY A Y\UX) 

<Ry + min{i?x, i?y} + 4(5 
U) + IviX A Y\U) + IviY A Y\UX) 

<Ry + min{i?x, i?y} + 4(5 
U) + IviX AY\U) + IviXY A XY\U) 

< Rx + Ry + m\n{Rx,RY} + 56 
U) + IviX AY\U) + IviXY A XY\U) 

< Rx + Ry + min{i?x, + 56 



(50) 



Proof. Let (C^, Cy) be the collections of codewords whose existence is asserted 
in lemma [3] Consider any Xi,Xfe e C^, and y^, y/ G Cy- Let us call their joint 
empirical distribution of (u, x^, y^ , x^. y^) as V^^j^y^^y (u, x, i, y). Using (|4T|) . 
and the fact that (u,Xi,yj) S Tv^xy^ 



Therefore, 



2 <- 2-n[^'(V)-min{flx,fly}-3(S] 



IviX A Y\U) < mm{Rx,RY} + 3(5 



(51) 



(52) 



Similarly, using the empirical distribution of (u, x^, y^), (u, Xfc, y^), and (u, x^, y^), 
we conclude that 



IviX A Y\U) < mm{Rx,RY} + 3(5 
IviX A Y\U) < mm{Rx,RY} + 3(5 
IviX A Y\U) < mm{Rx,RY} + 3(5 

Since (u, x,, y^-, x^) G Tv^^yx^ 



(u,Xj,y-,Xfc) 



(53) 
(54) 
(55) 



(56) 



Using (|56p . and the upper bound we obtained in (|^^ . 

IviX A Y\U) + IviX AY\U)+ IviX A X\UY) 
< Rx + mm{Rx,RY} + 4(5 



(57) 
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Similarly, since (u, x^, yj, x^:) G '^v^^yx' ^® conclude that 

IviX A Y\U) + IviX A Y\U) + A X\UY) 

< Rx + inm{Rx , Ry} + 4(5 (58) 

By a similar argument for the empirical distribution of (u, x;, y^, y^), (u, x/j, y^ , yj), 
and using the upper bound we obtained in (|43p , the following would respectively 
be concluded 

Iv{X A Y\U) + Iv{X A Y\U) + Iv{Y A Y\UX) 

< i?y + min{ i?x , i?y } + 4(5 (59) 
/v'(X A Y\U) + /y(X A y|C/) + Iv{Y A y|c/x) 

< i?y + min{i?x, i?y} + 4(5 (60) 

Finally, using the empirical distribution of (u, x^, y^ , x^, y^), (u, x^, y^, x^, y^), 
and the upper bound in (|44)l . 



iv{x A Y\u) + iv{x A y|[/) + iv{XY A xy|c/) 

< i?x + i?y + + min{i?x, + 55 (61) 
/y(X A Y\U) + Iv{X A Y\U) + Iv{XY A lr|C/) 

<i?x+i?y + +min{i?x,i?y} + 55. (62) 

□ 

Theorem 1. For every finite setU, VxYU G Vn{XxyxU) such that X — U — Y 
, Rx > 0, Ry > , (5 > 0, and u G Jp^, t/iere exists a multi-user code 

C = {(a;,, y^-. A,) : i = 1, ...A/i, J = 1, -M^.} (63) 

wit/i a;, G Tp^|,,(«), G Tp^|„(u) for all i and j, > and 

M* > 2"(-R^-*), such that for every MAC W : X x y ^ Z 

e{C,W) < 2^"-^^^^^^^'^^'^'^^^"^~^^ (64) 

whenever n > ni{\Z\, \y\, \U\,S), where 

E,,{Rx,Ry,W,Pxyu) 

- p=X^XY ' ' ^' ^^^^^ 
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and Ei3{Rx, Ry , W, Pxyu), = ^tY, XY are defined respectively by 

Ex{Rx,Ry,W,Pxyu) = 

min D{VzixYu\\W\PxYu) + Iv{X /\Y\U) 

+ \I{X AXZ\YU) + IviX AY\U)-Rx\^ (66) 

Ey{Rx,Ry,W,Pxyu) = 

min D(VzixYu\\W\PxYu)+Iv{X AY\U) 

+ \IiY AYZ\XU) + IviX AY\U)-Ry\+ (67) 

Exy{Rx,Ry,W,Pxyu) = 

min D{VzixYu\\W\PxYu)+Iv{X AY\U) 

'^UXYXYZ^^XY 

+ \I{XY A XYZ\U) + Iv{X A Y\U) - Rx - (68) 

where 

Vx = {VuxYxz ■ a{VuxYz) > a{V^xYz) 

and yjjjj-y^safiis/ies the relevant conditions in Lemma^ 

Vi- = {VuxYYZ ■ a{VuxYz) > aiV^xYz) 

and yjjj^yy. saiis/ies the relevant conditions in Lemma^^ 

VxY = {VfjxYXYZ ■ o^i^uxYz) > aiV^xYz) 

and yjjjj-y^y satis/ies all the conditions in Lemma^ (69) 

Remark 1. This exponential error bound can he universally obtained for all 
MAC's with given input and output alphabets. Note, it is a universal bound since 
the choice of the codewords does not depend on the channel, and the decoding 
rule is independent of the channel statistics. 

Proof Fix U, VxYU & Vn{X x y x U) with X - U - Y , Rx > Q, Ry > Q, 
(5 > 0, and u e T^^. Let C*x = {xi,X2, ...,xa/|.} and Cy = {yi,y2, •■•,yA/* } be 
the cohections of codewords whose existence is asserted in lemma |3l Consider 
the muhiuser code 

C = {(x„y^., Aj) : * - 1, ...Mi, J - 1, ...M^} (70) 

where the Dij are a-decoding sets for u. Taking into account the given u, the 
a-decoding yields the decoding sets 

D,j = {z : a(u,x.,,yj , z) < q;(u, x^, y,, z) for all(fc,0 ^ (i,j)} 
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The average probability of this muhiuser code can be written as 

+ i^E^"(U^-i-^'y.) (71) 

The first term on the right side can be written as 



for some 7^ i}|u, x^, 

^ 2-"[-0(^z|xyc/l|W|yxi'r/)+-f^v'(2|Xl'C/)] 



El-r,, (u,Xi,y.) 



.|{z : (u,x,;,y^,Xfc,z) E Ty^^^^Jor some fc 7^ i}\ (72) 
The second term in ((72)) can be upper bounded by 



M*M* 



•|{z : (u,x,,y^,Xfe,z) £ Ty^^^^^^for some fc 7^ i}\ 



M*M* 

^ " i,3 k-^i 

By properties of the codewords, mentioned in packing lemma, we can bound 
the right side of ([75)1 by 

<&^v{-n{Fx{V)-lly{Z\UXYX)W (74) 

By simple calculation, the exponent in (|74p can be rewritten as 

Iv(X A Y\V) +Iv{X A Y\U) + /y(l A XZ\UY) 

-Hv{Z\UXY) - Rx (75) 
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By using the fact that On the other hand, by the property of the codebook, the 
following bound for the second term, on the right side of ([72]) . can be obtained 



.|{z : (u,Xi,y^,Xfc,z) £ Ty^^y^^for some k ^ i)\ 
< exp ( - n[Iv{X A Y\U) - Hy{Z\UXY) - 3(5]) (76) 



By combining the exponents of (|74|) and ([TB]) . the right side of (f72|) can be 
bounded by 

where Ex{Rx , Ry,W, Pxyu) is defined in ([66)) . Similarly, by using a similar 
argument for the second term on the right side of (|7ip . we can show that 

^^^^^W^"(|J Adx.,y,) < 2-"^-(^-«-'^'^--) (78) 

where Ey{Rx , Ry ,W, Pxyu) is defined in ([67)1 . Now, consider the third term 
on the right side of (|7T1) . It can be written as 



l-—J2w'^(^{z : a(u,Xi,yj-,z) > a(u, Xfc, y,, z), 
for some {k,l) ^ (i, j)}|u,Xi,y^J 

^ 2'nlD{VzfxYu\\W\VxYu)+Hv(Z\XYU)] 

XYZ^^XY 



.|{z: (u,x„yj.,Xfe,y;,z) GTy^^^^^^ 

for some {k,l) ^ (i, j)}| (79) 
The second term in ([7^ can be upper bounded by 

.|{z : z e rv^|^^^^^(u,x„yj-,Xfc,y,)}| (80) 

The second term is actually the cardinality of Ty^^^^^^^ (u, x^, y^, Xfe, yj, which 

is equal to exp {ni/y (Z|J7XyXF)}. By the properties of the codewords, the 
first term in ([75]) can be upper bounded by exp {— ri[i^xy (V^)]}- Therefore l|80p 
can be bounded by 

< exp{-n[FYy(V") - By {Z\U XY XY)\} (81) 
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By simple calculation, the exponent in ([5T|) can be rewritten as 

Iv{X A Y\U) + Iv{X A Y\U) + IviXY A XYZ\U) 

-Hv{Z\UXY) -Rx-Ry (82) 

By using the properties of codewords, the following bound for the second term 
on the right right side of ([79|) can be obtained as follows 



M*M^ 



|{z : (u,Xj,y^,Xfc,y;,z) G Ty^^^it^Jov some (k,l) ^ (i, j)}| 

< exp ( - n[Iv{X A Y\U) - Hv{Z\UXY) - 3(5]) (83) 

By combining the exponents of ([5T|) and ([55]) . the right side of ([75]) can be 
bounded by 

^ 2-nExY(Rx,RY ,W,PxYu) ^g4^ 



where £;xY(i?x, W', Pxyu) is defined in ([Ml)- Now, it follows from ((77l) . ((78)) . 
and ([M)) , that the average probability of the given code is upper bounded by 

e(C, W) < 2-«[^-(-R^'«^>>^>^^^^)-'51 (85) 
where Eex{Rx, Ry,W, Pxyu) is defined in (|55l) . □ 

In the following, we prove that the random coding bound in theorem [1] will 
result in a tighter bound in comparison to the best known random coding bound, 
found in [12]. For this purpose, let us use the minimum equivocation decoding 
rule. 

Definition 3. Given u, for a multiuser code 

C = {{x,, y^,D,,) : * = 1, ...Mi,j = 1, ...M^.} 

we say that the Dij are minimum equivocation decoding sets for u if z Cz Dij 
implies 

H{xiyAzu) = m:niH{xkyi\zu). 

•' k,l 

It can be easily observed that these sets are equivalent to a-decoding sets, where 
a{u, X, y, z) is defined as 

a{VuxYz)=Hv{XY\ZU). (86) 

Here, VjjXYZ is the joint empirical distribution of (tt, x, y, z). 

Theorem 2. For every finite set U, Vxyu G ^(^) : Rx > 0, i?y > 0, and 
W : X xy -> Z, 

Ef3 {Rx , Ry , W, Pxyu ) > E^f,{Rx,RY,W, Pxyu) 

P^X,Y, XY (87) 
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Hence 

E,,{Rx,Ry,W,Pxyu) > E^{Rx,Ry.W,Pxyu) 

for all PxYU G ^(^) satisfying X — U — Y . Here, is the random coding 
exponent of [12]. E^^ are also defined in [12]. 

Proof. For any v„xyxz G Vx, 

Hv{XY\ZU) > Hv{XY\ZU), (89) 

therefore, by subtracting Hv{Y\ZU) form both sides of ([55]) . we can conclude 
that 

Hv{X\U) - Iv{X A YZ\U) > Hv{X\U) - A YZ\U), 

Since Vxu — Vxij — Pxu, the last inequality is equivalent to 

Iv{X A YZ\U) < Iv{X A YZ\U) 

Since Iv{X A XZ\YU) + I{X A Y\U) > Iv{X A YZ\U), it can be seen that for 
any v-^^^^z g Vx 

Iv{X A XZ\YU) + I{X A Y\U) > Iv{X A Y Z\U) 

Moreover, since 

Vx C {VuxYxz ■ ^uxYZ e V{Puxy) 

I{X AY\U) < Rx +36} (90) 

it can be easily concluded that 

Ex{Rx,Ry, W, Pxyu) > E[^xiRx,RY, W, Pxyu)- 
Similarly, for any Vjj^yyz ^ ^ 

Hv{XY\ZU) > Hv{XY\ZU). 
By using the fact that, Vyu = ^yu ^ Pyu, it can be concluded that 

Iv{Y A YZ\XU) + I{X A Y\U) > Iv{Y A XZ\U). 

Since 

Vy C {VjjxYYZ ■ VUXYZ € V{Puxy) 

I{X AY\U) < Rx + S6} (91) 

we conclude that 

Ey{Rx.Ry, W, Pxyu) > E[^y{Rx,Ry, W, Pxyu)- 
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Similarly, we can conclude that, for any Vjjxyxyz ^ ^xy, 

Iv{XY A XYZ\U) + I{X A Y\U) > Iv{XY A Z\U) + I{X A Y\U). 

Since 

VxY C {VuXYXYZ ■ ^UXYZ G V{PuXy) 

I{X AY\U) < Rx + (92) 

it can be concluded that 

Exy{Rx,Ry, W, Pxyu) > E^xy{Rx,Ry, W, Pxyu)- 

□ 

The last theorem shows that Eg^iRx , Ry ,W, Pxyu) is at least as large as 
the Liu, Hughes [12] exponent. In the following, wc show that at low rate pairs, 
we may have a strictly better result. To illustrate this, let us focus on the 
case where both codebooks have rate zero, Rx = Ry = 0. For small S, any 
^UXYXZ ^ will satisfy the following relationships 

X-U-Y, X-U-Y, X-UY-X (93) 

Therefore, any V^^yxz ^ '^^^ be written as 

VziuxyxPx\uPy\uPx\uPu. (94) 

Similarly, any Vjj-^yyz ^ written as 

^z\uxyyPx\uPy\uPy\uPu, (95) 

and any Vjj^yxYZ ^ ^xy can be written as 

^z\uxyxyPx\uPy\uPx\uPy\uPu- (96) 

For a moment, let us consider the point to point data communication. By 
using only a random coding argument, and without any expurgation, one can 
prove the following result. 

Lemma 5. For every R > 0, S > and every type of P £ Pn{X) satisfying 
H{P) > R, there exist M > 2"(^~'') sequences in Tp such that for every Pxx ^ 
V{X X X), 

i=l k^i 

provided that n > no{\X\, \y\,6). 
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Now, let us multiply both sides of (|97p by M. It can be shown that for every 
l<i < M, 

By using these sequences as our set of codewords, and using a-decoding, we 
will end up with a result very similar to [3]. The only difference is that our 
minimization would be taken over all distributions satisfying I{X A X) < 2R, 
instead of I{X AX) < R. Using the appropriate decoding rule, this bound would 
be exactly the same as the typical random coding bound that Barg and Forney 
found in [2]. As we can see, in point to point communications, even without 
doing any expurgation, we ended up with a strictly better bound in comparison 
to the usual random coding bound. Needless to say that if we eliminate half of 
the codewords in (P7|). the result would be equal to the expurgated bound [3]. 
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